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DETAILED ACTION 

Response to Amendment 

1 . The Examiner acknowledges Applicant's submission on 1/7/08 including 
amendments to claims 1 and 56. All pending claims have been rejected where the 
previous rejection has been maintained. In view of the amendments, the 35 U.S.C. 1 1 2 
rejections against claims 1, 6, 7, 9 and 10 have been withdrawn. In view of arguments 
against the 35 U.S.C. 112 rejection against claim 1 1, the arguments have been 
persuasive and the rejection against claim 1 1 has been withdrawn. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1 -7, 9-34, 36, 37, 39-56 and 58-86 are rejected under 35 U.S.C. 1 03(a) 
as being unpatentable over "Adjustable Filmstrips and Skims as Abstractions for a 
Digital Video Library" (Christel et al.), herein referred to as Christel, "Bayesian Modeling 
Of Video Editing And Structure: Semantic Features For Video Summarization And 
Browsing" (Vasconcelos et al.), herein referred to as Vasconcelos and U. S. Patent No. 
6, 880, 171 B1 (Ahmad et al.), herein referred to as Ahmad. 

Referring to claim 1 , Christel discloses a method of presenting information 
regarding a video with a plurality of frames (page 2, column 2, lines 1 1 -1 9). Christel 
discloses summarizing a video by identifying a plurality of segments of the video based 
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upon an event (page 4, column 2, lines 28-32). Christel discloses that each of the 
segments includes a plurality of sequential frames of the video (page 4, column 2, lines 
8-20). Christel discloses displaying a summarization in a first portion of a display 
(Figure 5), wherein the first portion discloses a snapshot of a skim and indicating the 
sequences that make up the skim at the bottom of the window. Christel discloses 
displaying a graphical user interface on a second portion of the display, sequentially 
indicating the relative location of the plurality of segments within the summarization 
relative to at least one other of the segments as each of the plurality of segments is 
displayed (Figure 5), wherein the second portion displays the match locations in relation 
to the location of plurality of segments within the summarization. The plurality of 
segments is bounded blocks of spatial region that are displayed on the second display 
portion of the display. Christel discloses displaying to the user the relative location for a 
first type of content of the video using a first visual indication (Figure 5), wherein this is 
represented as the gray blocks at the bottom of the window and displaying the relative 
location for second type of content of the video using a second visual indication different 
from the first visual indication (Figure 5), wherein the white lines showing the match 
locations represents the second visual indication. Christel discloses receiving from the 
user through interaction with the graphical user interface a selection of one of the 
plurality of segments (Figure 6 and page 5, column 2, lines 1-7). The user interaction 
allows for viewing desired segment based on the initial summary of the segment that 
the user has viewed. In response to the selection, presenting a selected one of the 
plurality of segments and not presenting at least one other of the plurality of segments. 
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Once the user has determined that a certain segment is to be viewed, user interaction 
with the display of Figure 6 allows for that distinct segment to be viewed while not 
viewing other segments, which can be skimmed through. Christel does not explicitly 
disclose that the event being captured is characterized by a play, wherein this play is of 
a sport. It would have been obvious for one skilled in the art at the time of the invention 
that the event being captured is characterized by a play within a sport. Christel includes 
all types of video when discussing capturing of important events from a video, wherein 
based on the content of the video, any important events relevant to that specific video 
would be captured. Christel discloses that the goal of the invention is to display the 
essential content of a video, and wherein it is obvious that when the video is that of a 
sporting event, the essential content would be the plays and hence it would be obvious 
for Christel to captures these plays, wherein these plays are those that are part of a 
sporting event. The plays clearly represent characterizations of semantic events of the 
sporting event, representing an event that has clear association with the context of the 
sporting event and describing specific events that have occurred. Christel does not 
disclose two different characterizations of semantic events, which are indicated by two 
different visual means. Vasconcelos discloses identifying specific semantic events and 
displaying the identified data through visual indications in a timeline (Figure 2). It would 
have been obvious to one skilled in the art at the time of the invention to learn from 
Vasconcelos to identify semantic events and display the identified data in a timeline. 
With Vasconcelos teaching that identification of semantic data will allow users to quickly 
access relevant and desired portions of a large video (page 5, column 1, lines 10-12) 
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provides a motivation for one skilled in the art to learn from Vasconcelos. Therefore, 
based on the information provided by Vasconcelos, Christel's video system would have 
been motivated to learn from Vasconcelos to identify semantic events from within video 
data. One skilled in the art at the time of the invention would have been motivated to 
learn from Vasconcelos to identify semantic events from video data and display the data 
in a timeline format. 

Christel and Vasconcelos do not disclose displaying the identified semantic events in 
the timeline with different visual indications for two different characterizations of 
semantic event. Ahmad discloses obtaining summary data related to video and audio 
data, with visual means for indicating a relative location for a first characterization of 
data in a video data and displaying the relative location for a second characterization of 
data in the video using a second visual indication different from the first visual indication 
(column 16, lines 54-65). It would have been obvious for one skilled in the art, at the 
time of the invention to learn from Ahmad to teach two different visual indications for two 
different characterizations or descriptions of information. Christel, Vasconcelos and 
Ahmad disclose systems for summarizing and obtaining relevant data from video and 
audio data, further displaying timelines for indicating relevant data. Ahmad further 
displays detailed bar information to indicate different visual means for different 
characterizations of data. It would have been obvious for one skilled in the art, at the 
time of the invention to learn from Ahmad to implement detailed bar data to further 
provide more detailed information concerning the video and summary data obtained. 
Vasconcelos discloses an objective for providing semantic information so as to display 
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data without overwhelming the user. Vasconcelos' timeline includes the identification of 
different characterizations of semantic events through letters A, D, C and S would 
further be efficiently handled by providing two different visual indications for different 
characterizations of semantic events. The display means of Ahmad would allow for 
easier accessing of desired data with clear displays mean for quickly identifying specific 
data. Therefore, one skilled in the art would have been motivated to learn from Ahmad 
to use different visual indications for displaying different characterizations of semantic 
events. 

Referring to claim 2, Christel, Vasconcelos and Ahmad disclose that the first and 
second semantic characterizations of the play temporally overlap in the summarization 
(Christel, Figures 5 and 6), where the summarization sections discloses two types of 
data with each of them temporally overlapping at distinct points. 

Referring to claim 3, Christel discloses a generally rectangular region where each of 

the plurality of segments is indicated within the generally rectangular region (Figure 5). 

Referring to claim 4, Christel discloses that the size of each of the plurality of 
segments is indicated in a manner such that the plurality of segments with a greater 
number of frames are larger than the plurality of segments with a lesser number of 
frames (Figure 5). 

Referring to claim 5, Christel discloses that the size of the regions between each 
of the plurality of segments is indicated in a manner such that the regions between with 
a greater number of frames are larger than plurality of segments with a lesser number of 
frames (Figure 5). 
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Referring to claim 6, Christel discloses that the user selects one of the plurality of 
segments by interacting with the graphical user interface at point within the displayed 
bounded spatial region associated with the selected one of the plurality of segments 
(Figure 6). The user selects one of the desired segments through the user interface of 
Figure 6, the user selector selected at a distinct point selects the desired one of the 
plurality of segments. 

Referring to claim 7, Christel discloses presentation of a selected one of the 
plurality of segments begins at a first frame of the segment irrespective of which point 
within the displayed bounded spatial region that the user interacted with (Figure 6 and 
page 5, column 2, lines 1-7). 

Referring to claim 9, Christel discloses that the presentation of a selected one of 
the plurality of segments begins at a frame of the segment temporally corresponding to 
the point within the displayed spatial region that the user interacted with (Figure 6). The 
segment selected begins at a first frame of the selected segment with all segments 
being temporally placed in relation to the bound spatial region. 

Referring to claim 10, Christel discloses including a selector by which the user 
may alternatively select a chosen of the presentation of a selected one of the plurality of 
segments beginning at the first frame of the segment irrespective of which point with the 
displayed bounded spatial region that the user interacted with (Figure 6 and page 5, 
column 2, lines 1-7). Christel also discloses presentation of a selected one of the 
plurality of segments beginning at a frame of the segment temporally corresponding to 
the point within the displayed bounded spatial region that the user interacted that the 
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user interacted with (Figure 6). The segment selected begins at a first frame of the 
selected segment with all segments being temporally placed in relation to the bound 
spatial region. 

Referring to claim 1 1 , Christel discloses including a user moveable scroll bar on 
the graphical user interface indicating the relative temporal location of the currently 
presented frames of the summary (Figures 5 and 6). Christel discloses that the user 
selects one of the plurality of segments by moving the scroll bar over the selected one 
of the plurality of segments with the scroll bar snapping to the beginning of the selected 
one of the plurality of segments (Figures 5 and 6). 

Referring to claim 12, Christel discloses that at least two of the plurality of 
segments is temporally overlapping (page 4, column 2, lines 26-29). 

Referring to claim 13, Christel discloses that the temporally overlapping 
segments are visually indicated in a manner such that each of the overlapping 
segments is independently identifiable (page 4, column 2, lines 26-31). 

Referring to claim 14, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the pluralities of segments (Figure 6). 

Referring to claim 15, Christel discloses that one of the pluralities of segments is 
the segment most temporally adjacent to the portion of the video (Figure 6). 

Referring to claim 1 6, Christel discloses that one of the pluralities of segments is 
the next temporally related segment (Figure 6). 
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Referring to claim 17, Christel discloses that the plurality of segments is the 
previous temporally related segment (Figure 6). 

Referring to claim 18, Christel discloses that a user selects a portion of the video 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video from the start thereof (page 5, column 1 , lines 5-1 1 ). 

Referring to claim 19, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments (page 5, column 1 , 
lines 5-1 1 and Figure 6). 

Referring to claim 20, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments starting from the 
beginning thereof (page 5, column 1 , lines 5-1 1 and Figure 6). 

Referring to claim 21 , Christel discloses that a user selects a portion of video not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
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video not included within the plurality of segments, and wherein the user selects a 
portion of the video included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the video within the plurality 
of segments (page 5, column 1 , lines 5-1 1 and Figure 6). 

Referring to claim 22, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 23, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
maintain the differentiation of the overlapping time periods (Figure5 and page 4, column 
2, lines 26-31). 

Referring to claim 24, Christel discloses that the temporal information is 
displayed within a time line, wherein the temporal information is presented in a plurality 
of layers in a direction perpendicular to the length of the time line (Figure 5). 

Referring to claim 25, Christel discloses that the temporal information is 
displayed within a time line, wherein textual information is included within the time line 
(Figure 6). 

Referring to claim 26, Christel discloses that additional textual information is 
displayed upon selecting a portion of the time line (Figure 6). 

Referring to claim 27, Christel discloses that temporal information is displayed 
together with a time line, wherein additional textual information is displayed together 
with selecting a portion of the time line (Figure 6). 
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Referring to claim 28, Christel discloses that temporal information is displayed 
within a time line, wherein additional audio annotation is presented upon presenting a 
portion of the time line (page 4, column 1, lines 33-37 and Figure 6). 

Referring to claim 29, Christel discloses a method of presenting information 
regarding a video with a plurality of frames (page 2, column 2, lines 11-19). Christel 
discloses identifying a plurality of different segments of the video, where each of the 
segments includes a plurality of frames of the video (page 4, column 2, lines 8-32). 
Christel discloses displaying, simultaneously with a segment of the video, a graphical 
user interface including information regarding the temporal location of the segments, 
relative to at least one other of the segments of the video (Figure 5). Christel discloses 
displaying in an interactive display a temporal location for a first characterization of the 
content of the video using a first visual indication (Figure 5), wherein this is represented 
as the gray blocks at the bottom of the window and displaying the temporal location for 
second characterization of the content of the video using a second visual indication 
different from the first visual indication (Figure 5), wherein the white lines showing the 
match locations represents the second visual indication. Christel discloses displaying to 
a user a selector by which the user may interact with the interactive display to select for 
viewing selective identified ones of the plurality of segments (Figure 6 and page 5, 
column 2, lines 1-7). The selector of the user interface of Figure 6 allows for the user to 
select for viewing one of the desired segments of the plurality of segments. Christel 
also discloses receiving user-selections of identified ones of the plurality of segments 
(page 5, column 2, lines 1-7) and presenting these segments (Figure 6). Christel does 
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not disclose two different characterizations of semantic events through indicated by two 
different visual means. Vasconcelos discloses identifying specific semantic events and 
displaying the identified data through visual indications in a timeline (Figure 2). It would 
have been obvious to one skilled in the art at the time of the invention to learn from 
Vasconcelos to identify semantic events and display the identified data in a timeline. 
With Vasconcelos teaching that identification of semantic data will allow users to quickly 
access relevant and desired portions of a large video (page 5, column 1, lines 10-12) 
provides a motivation for one skilled in the art to learn from Vasconcelos. Therefore, 
based on the information provided by Vasconcelos, Christel's video system would have 
been motivated to learn from Vasconcelos to identify semantic events from within video 
data. One skilled in the art at the time of the invention would have been motivated to 
learn from Vasconcelos to identify semantic events from video data and display the data 
in a timeline format. 

Christel and Vasconcelos do not disclose displaying the identified semantic 
events in the timeline with different visual indications for two different types of semantic 
event. Ahmad discloses obtaining summary data related to video and audio data, with 
visual means for indicating a relative location for a first type of data in a video data and 
displaying the relative location for a second type of data in the video using a second 
visual indication different from the first visual indication (column 16, lines 54-65). It 
would have been obvious for one skilled in the art, at the time of the invention to learn 
from Ahmad to teach two different visual indications for two different types of 
information. Christel, Vasconcelos and Ahmad disclose systems for summarizing and 
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obtaining relevant data from video and audio data, further displaying timelines for 
indicating relevant data. Ahmad further displays detailed bar information to indicate 
different visual means for different types of data. It would have been obvious for one 
skilled in the art, at the time of the invention to learn from Ahmad to implement detailed 
bar data to further provide more detailed information concerning the video and summary 
data obtained. Vasconcelos discloses an objective for providing semantic information 
so as to display data without overwhelming the user. Vasconcelos' timeline includes the 
identification of different types of semantic events through letters A, D, C and S would 
further be efficiently handled by providing two different visual indications for different 
types of semantic events. The display means of Ahmad would allow for easier 
accessing of desired data with clear displays mean for quickly identifying specific data. 
Therefore, one skilled in the art would have been motivated to learn from Ahmad to use 
different visual indications for displaying different types of semantic events. 

Referring to claim 30, Christel discloses a generally rectangular region where 
each of the plurality of segments is indicated within the generally rectangular region 
(Figure 5). 

Referring to claim 31 , Christel discloses that the size of each of the plurality of 
segments is indicated in a manner such that the plurality of segments with a greater 
number of frames are larger than the plurality of segments with a lesser number of 
frames (Figure 5). 

Referring to claim 32, Christel discloses that the size of the regions between 
each of the plurality of segments is indicated in a manner such that the regions between 
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with a greater number of frames are larger than plurality of segments with a lesser 
number of frames (Figure 5). 

Referring to claim 33, Christel discloses an indicator that indicates the current 
position within the temporal information of a currently displayed portion of the video 
(Figure 5). 

Referring to claim 34, Christel discloses the indicator changes location relative to 
the temporal information as the portion of the currently displayed portion of the video 
changes (Figure 5). 

Referring to claim 36, Christel discloses indicating with an indicator the current 
position within the temporal information of a currently displayed portion of said video 
and modifying the position of the indicator within the temporal information, which 
modifies the displayed portion of the video (Figures 5 and 6). 

Referring to claim 37, Christel discloses that the indicator is modified to a portion 
of the video that is not included within the plurality of segments (Figures 5 and 6). 

Referring to claim 38, Christel discloses the first type of content and second type 
of content are selectable by a user for presentation to the user (Figure 5, page 5, 
column 1, lines 9-12 and Figure 6). 

Referring to claims 39 and 60, Christel discloses that at least two of the plurality 
of segments is temporally overlapping (page 4, column 2, lines 26-29). 

Referring to claims 40 and 61 , Christel discloses that the temporally overlapping 
segments are visually indicated in a manner such that each of the overlapping 
segments is independently identifiable (page 4, column 2, lines 26-31). 
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Referring to claim 41, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the pluralities of segments (Figure 6). 

Referring to claim 42, Christel discloses that one of the pluralities of segments is 
the segment most temporally adjacent to the portion of the video (Figure 6). 

Referring to claim 43, Christel discloses that one of the pluralities of segments is 
the next temporally related segment (Figure 6). 

Referring to claim 44, Christel discloses that the plurality of segments is the 
previous temporally related segment (Figure 6). 

Referring to claim 45, Christel discloses that a user selects a portion of the video 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video from the start thereof (page 5, column 1 , lines 5-1 1 ). 

Referring to claim 46, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments (page 5, column 1 , 
lines 5-1 1 and Figure 6). 

Referring to claim 47, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
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presents the portion of the video within the plurality of segments starting from the 
beginning thereof (page 5, column 1, lines 5-11 and Figure 6). 

Referring to claim 48, Christel discloses that a user selects a portion of video not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
video not included within the plurality of segments, and wherein the user selects a 
portion of the video included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the video within the plurality of segments 
(page 5, column 1 , lines 5-1 1 and Figure 6). 

Referring to claim 49, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 50, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
maintain the differentiation of the overlapping time periods (Figure5 and page 4, column 
2, lines 26-31). 

Referring to claims 51 and 72, Christel discloses that the temporal information is 
displayed within a time line, wherein the temporal information is presented in a plurality 
of layers in a direction perpendicular to the length of the time line (Figure 5). 
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Referring to claims 52 and 73, Christel discloses that the temporal information is 
displayed within a time line, wherein textual information is included within the time line 
(Figure 6). 

Referring to claims 53 and 74, Christel discloses that additional textual 
information is displayed upon selecting a portion of the time line (Figure 6). 

Referring to claims 54 and 75, Christel discloses that temporal information is 
displayed together with a time line, wherein additional textual information is displayed 
together with selecting a portion of the time line (Figure 6). 

Referring to claims 55 and 76, Christel discloses that temporal information is 
displayed within a time line, wherein additional audio annotation is presented upon 
presenting a portion of the time line (page 4, column 1, lines 33-37 and Figure 6). 

Referring to claim 56, Christel discloses a method of presenting information 
regarding audio (page 2, column 2, lines 11-19). Christel discloses identifying a plurality 
of different segments of the audio, where each of the segments includes a temporal 
duration of the audio (page 4, column 2, lines 8-32). Christel discloses displaying, 
simultaneously with the segment of the audio, a graphical user interface including 
information regarding the temporal location of the segments, relative to at least one 
other of segment of the audio (Figure 5). Christel discloses displaying in an interactive 
display the temporal location for a first characterization of content of the audio using a 
first visual indication (Figure 5), wherein this is represented as the gray blocks at the 
bottom of the window and displaying the temporal location for second characterization 
of content of the audio using a second visual indication different from the first visual 
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indication (Figure 5), wherein the white lines showing the match locations represents 
the second visual indication. Christel discloses displaying to a user a selector by which 
the user may interact with the interactive display to select for listening selective 
identified ones of the plurality of segments (Figure 6 and page 5, column 2, lines 1-7). 
The selector of the user interface of Figure 6 allows for the user to select for viewing 
and listening one of the desired segments of the plurality of segments of video data. 
Video data includes data that can be viewed and listened to. Christel also discloses 
receiving user-selections of identified ones of the plurality of segments (page 5, column 
2, lines 1-7) and presenting these segments (Figure 6). Christel does not disclose two 
different characterizations of semantic events through indicated by two different visual 
means. Vasconcelos discloses identifying specific semantic events and displaying the 
identified data through visual indications in a timeline (Figure 2). It would have been 
obvious to one skilled in the art at the time of the invention to learn from Vasconcelos to 
identify semantic events and display the identified data in a timeline. With Vasconcelos 
teaching that identification of semantic data will allow users to quickly access relevant 
and desired portions of a large video (page 5, column 1, lines 10-12) provides a 
motivation for one skilled in the art to learn from Vasconcelos. Therefore, based on the 
information provided by Vasconcelos, Christel's video system would have been 
motivated to learn from Vasconcelos to identify semantic events from within video data. 
One skilled in the art at the time of the invention would have been motivated to learn 
from Vasconcelos to identify semantic events from video data and display the data in a 
timeline format. 
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Christel and Vasconcelos do not disclose displaying the identified semantic 
events in the timeline with different visual indications for two different types of semantic 
event. Ahmad discloses obtaining summary data related to video and audio data, with 
visual means for indicating a relative location for a first type of data in a video data and 
displaying the relative location for a second type of data in the video using a second 
visual indication different from the first visual indication (column 16, lines 54-65). It 
would have been obvious for one skilled in the art, at the time of the invention to learn 
from Ahmad to teach two different visual indications for two different types of 
information. Christel, Vasconcelos and Ahmad disclose systems for summarizing and 
obtaining relevant data from video and audio data, further displaying timelines for 
indicating relevant data. Ahmad further displays detailed bar information to indicate 
different visual means for different types of data. It would have been obvious for one 
skilled in the art, at the time of the invention to learn from Ahmad to implement detailed 
bar data to further provide more detailed information concerning the video and summary 
data obtained. Vasconcelos discloses an objective for providing semantic information 
so as to display data without overwhelming the user. Vasconcelos' timeline includes the 
identification of different types of semantic events through letters A, D, C and S would 
further be efficiently handled by providing two different visual indications for different 
types of semantic events. The display means of Ahmad would allow for easier 
accessing of desired data with clear displays mean for quickly identifying specific data. 
Therefore, one skilled in the art would have been motivated to learn from Ahmad to use 
different visual indications for displaying different types of semantic events. 
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Referring to claim 58, Christel discloses indicating with an indicator the current 
position within the temporal information of a currently displayed portion of said audio 
and modifying the position of the indicator within the temporal information, which 
modifies the displayed portion of the audio (Figures 5 and 6). 

Referring to claim 59, Christel discloses that the indicator is modified to a portion 
of the audio that is not included within the plurality of segments (Figures 5 and 6). 

Referring to claim 62, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the pluralities of segments (Figure 6). 

Referring to claim 63, Christel discloses that one of the pluralities of segments is 
the segment most temporally adjacent to the portion of the audio (Figure 6). 

Referring to claim 64, Christel discloses that one of the pluralities of segments is 
the next temporally related segment (Figure 6). 

Referring to claim 65, Christel discloses that the plurality of segments is 
the previous temporally related segment (Figure 6). 

Referring to claim 66, Christel discloses that a user selects a portion of the audio 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio from the start thereof (page 5, column 1 , lines 5-1 1 ). 

Referring to claim 67, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
audio included within the plurality of segments, wherein in response thereto, the system 



Application/Control Number: 10/016,941 Page 21 

Art Unit: 2173 

presents the portion of the audio within the plurality of segments (page 5, column 1 , 
lines 5-11 and Figure 6). 

Referring to claim 68, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
audio included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio within the plurality of segments starting from the 
beginning thereof (page 5, column 1 , lines 5-1 1 and Figure 6). 

Referring to claim 69, Christel discloses that a user selects a portion of audio not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
audio not included within the plurality of segments, and wherein the user selects a 
portion of the audio included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the audio within the plurality of segments 
(page 5, column 1 , lines 5-1 1 and Figure 6). 

Referring to claim 70, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 71, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
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maintain the differentiation of the overlapping time periods (Figure5 and page 4, column 
2, lines 26-31). 

Referring to claim 77, Christel discloses a user selectable skip function skips a 
set of frames to a modified location of the video in at least one of a forward temporal 
direction or a reverse temporal direction, and displays the video at the modified location 
(Figure 6). 

Referring to claim 78, Christel discloses a user selectable skip function skips to a 
later temporal segment or a previous temporal segment, and displays video at the later 
temporal segment or the previous temporal segment, respectively (Figure 6). 

Referring to claim 79, Christel discloses that a user selectable scan function 
skips a set of frames to a modified location of the video in at least one of a forward 
temporal direction or a reverse temporal direction, and displays the video at the 
modified location, and thereafter automatically skips another set of frames to another 
modified location of the video in at least one of the forward temporal direction or the 
reverse temporal direction, and displays the video at another modified location (Figure 
6). 

Referring to claim 80, Christel discloses that at least one of the forward temporal 
direction and reverse temporal direction are consistent with the different segments 
(Figure 6). 

Referring to claim 81 , Christel discloses that the display of the video is at the 
start of the respective one of the different segments (Figure 6). 
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Referring to claim 82, Christel discloses that the display of the video is at a 
predetermined offset within the respective one of the different segments (Figure 6). 

Referring to claim 83, Christel discloses that the graphical user interface displays 
a respective image associated with at least a plurality of the different segments (Figure 
6). 

Referring to claim 84, Christel discloses that the respective image associated 
with the currently presented different segments is visually highlighted (Figure 6). 

Referring to claim 85, Christel discloses that during presentation of the video 
visually highlighted respective images are highlighted in a substantially regular interval 
while the sequence of the presentation of the video is at substantially irregular intervals 
(Figure 6). 

Referring to claim 86, Christel discloses presentation of the different segments 
may be modified by a plurality of different functions, and wherein the user may 
customize another function, not previously explicitly provided, by combing a plurality of 
the plurality of different functions into a single function (page 5, column 1 , lines 9-1 1 and 
Figure 6). 

Response to Arguments 

3. Applicant's arguments against the 35 U.S.C. 103 rejections have been fully 
considered but they are not persuasive. 

Applicant argues that Christel does not disclose selection of the bounded 
segments. In contrast to Applicant's arguments, Christel discloses Figure 6 that is 
generated in response to user selection is a result of transitioning from the Figure 5 to 
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Figure 6. Adjusting the compression ration involves selection of segments where the 
details of the segments are adjusted based on user selection. The same segments 
within the first and second display portions are selected and further adjusted to show 
details associated with those segments. Skim playback includes playing specific 
segments that have been identified and not playing segments that are part of the skim 
data. As shown in Figures 5 and 6, a scroll bar is displayed in the area with the 
segments which can be used to select distinct segments. User selection is made to 
adjust previous decisions made associated with generated segments. Christel 
discloses that Figure 6 is an adjusted interface based on further selections made from 
the user interface of Figure 5. The video data and the plurality of segments therefore 
are the same that are in Figure 5. The segment in Figure 5 is selected for the user to 
adjust in Figure 6. The bounded segments from Figure 5 are selected and adjusted. 
These segments are then played with exclusion of other video data segments that are 
not played. Playback of skim data involves playback of specified segments with 
exclusion of video segments that are not part of the skim data. 

Applicant argues that no cited art discloses the limitation displaying to a user the 
relative location for a first semantic characterization of a sports play in video using a first 
visual indication and displaying the relative location for a second semantic 
characterization of a sports play in the video using a second visual indication different 
from the first visual indication. As previously address, in contrast to Applicant's 
arguments, the combination of Christel, Vasconcelos and Ahmad discloses displaying to 
a user the relative location for a first semantic characterization of a sports play in video 
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using a first visual indication and displaying the relative location for a second semantic 
characterization of a sports play the video using a second visual indication different from 
the first visual indication. The three references above recite limitations that are claimed 
and are directed to visualization of media data while emphasizing specified data within 
the media data. The features are disclosed in the combination of Christel, Vasconcelos 
and Ahmad where these three references disclose the features claimed. The present 
invention is directed to claiming features that are disclosed previously in the field of 
media summarization. In view of the features disclosed in the three references, it would 
have been obvious to one skilled in the art at the time of the invention to implement 
displaying to a user the relative location for a first semantic characterization of a sports 
play in video using a first visual indication and displaying the relative location for a 
second semantic characterization of a sports play the video using a second visual 
indication different from the first visual indication. The teachings of Christel, 
Vasconcelos and Ahmad would have suggested to one of ordinary skill in the art that 
media data with certain characteristics is summarized with visual indications. The 
combination would have been obvious in view of what these three references would 
have suggested to one of ordinary skill in the art in the field of media summarization. As 
stated in Applicant's arguments, it would have been obvious that the video data that is 
summarized in Christel includes sports play data. Clearly Christel has conveyed that 
visual indication is provided in the bars on Figures 5 and 6 to identify distinct 
summarized clips. These visual indications disclose relative locations of distinct 
summarized clips in association with the entire video data. The Figures 5 and 6 of 
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Christel indicate the visual indications are relied upon to easily identify specific video 
data. Vasconcelos is a system that has the same objective to summarize video data 
specifically associated with semantic characterization of the content of the video data. 
The teachings of identifying specific video data based on semantic characterization 
would suggest to Christel which is also summarizing video data that the specific desired 
video clips that are identified in Christel can be based on semantic characterization 
which is associated with actual content of the video. Regardless of if Vasconcelos 
provides a technical means to carry out the semantic characterization, Vasconcelos 
does disclose identifying semantic characterizations which would suggest to Christel a 
means for identifying summarized video data based on the content of the video data. 

Conclusion 

4. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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Responses to this action should be submitted as per the options cited below: The 
United States Patent and Trademark Office requires most patent related 
correspondence to be: a) faxed to the Central Fax number (571-273-8300) b) hand 
carried or delivered to the Customer Service Window (located at the Randolph Building, 
401 Dulany Street, Alexandria, VA 22314), c) mailed to the mailing address set forth in 
37 CFR 1 .1 (e.g., P.O. Box 1450, Alexandria, VA 22313-1450), or d) transmitted to the 
Office using the Office's Electronic Filing System. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Namitha Pillai whose telephone number is (571) 272- 
4054. The examiner can normally be reached from 8:30 AM - 5:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Doon Chow can be reached on (571) 272-7767. 

All Internet e-mail communications will be made of record in the application file. 
PTO employees do not engage in Internet communications where there exists a 
possibility that sensitive information could be identified or exchanged unless the record 
includes a properly signed express waiver of the confidentiality requirements of 35 
U.S.C. 122. This is more clearly set forth in the Interim Internet Usage Policy published 
in the Official Gazette of the Patent and Trademark on February 25, 1997 at 1 195 OG 
89. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Group receptionist whose telephone number is 
(571)272-2100. 
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Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). 

Namitha Pillai 
Patent Examiner 
Art Unit 2173 
April 23, 2008 

/Tadesse Hailu/ 

Primary Examiner, Art Unit 2173 



